vtd: fix multiple Dom0 S3 on hosts that support Queued Invalidation.

On such hosts we can't do multiple Dom0 S3 when VT-d is enabled.
The cause is: during the first S3 resume, init_vtd_hw() initializes
the invalidation function pointers to the register-based ones and later
enable_qinval() forgets to overwrite the flush function pointers to
queued-based ones, so actually Queued Invalidaton is enabled, but we
actually use the register-based invalidation function! Later during
the second Dom0 S3, in iommu_suspend() -> iommu_flush_all(), we try to
use the register-based invalidation functions to perform global flush
while Queued Invalidation is enabled, and this can cause a host reset
because VT-d spec says: when the queued invalidation is enabled,
software must submit invalidation commands only through the IQ (and
not through any invalidation command registers).

The attached patch fixes the buggy enable_qinval(). And in
iommu_resume(), we invoke iommu_flush_all() for safety.

Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>

diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/vtd/iommu.c
index e2b026ca7e80a30007b75f6d529ddcad18dc1d5b..dc18b3fde5968372904642777cde55966d1d223c 100644 (file)
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -1953,16 +1953,34 @@ void iommu_resume(void)
 {
     struct acpi_drhd_unit *drhd;
     struct iommu *iommu;
+    struct iommu_flush *flush;
     u32 i;
 
     if ( !vtd_enabled )
         return;
 
+    /* Re-initialize the register-based flush functions.
+     * In iommu_flush_all(), we invoke iommu_flush_{context,iotlb}_global(),
+     * but at this point, on hosts that support QI(Queued Invalidation), QI
+     * hasn't been re-enabed yet, so for now let's use the register-based
+     * invalidation method before invoking init_vtd_hw().
+     */
+    if ( iommu_qinval )
+    {
+        for_each_drhd_unit ( drhd )
+        {
+            iommu = drhd->iommu;
+            flush = iommu_get_flush(iommu);
+            flush->context = flush_context_reg;
+            flush->iotlb = flush_iotlb_reg;
+        }
+    }
+
     /* Not sure whether the flush operation is required to meet iommu
      * specification. Note that BIOS also executes in S3 resume and iommu may
      * be touched again, so let us do the flush operation for safety.
      */
-    flush_all_cache();
+    iommu_flush_all();
 
     if ( init_vtd_hw() != 0  && force_iommu )
          panic("IOMMU setup failed, crash Xen for security purpose!\n");

diff --git a/xen/drivers/passthrough/vtd/qinval.c b/xen/drivers/passthrough/vtd/qinval.c
index e82337045f2f6d6e06c022815d94d6207651c474..60c7e0ad70f0bb61a6d9ceb7b79015349cee148a 100644 (file)
--- a/xen/drivers/passthrough/vtd/qinval.c
+++ b/xen/drivers/passthrough/vtd/qinval.c
@@ -432,10 +432,11 @@ int enable_qinval(struct iommu *iommu)
                     "Cannot allocate memory for qi_ctrl->qinval_maddr\n");
             return -ENOMEM;
         }
-        flush->context = flush_context_qi;
-        flush->iotlb = flush_iotlb_qi;
     }
 
+    flush->context = flush_context_qi;
+    flush->iotlb = flush_iotlb_qi;
+
     /* Setup Invalidation Queue Address(IQA) register with the
      * address of the page we just allocated.  QS field at
      * bits[2:0] to indicate size of queue is one 4KB page.